Add a build-time opt-in hybrid API documentation POC - #1874

Summary

Adds a build-time opt-in hybrid Astro/Starlight POC. With ASPIRE_HYBRID_API_DOCS=1, non-API documentation stays static and C# / TypeScript API HTML and Markdown render on demand using the official Node adapter. The default build and static Azure deployment remain unchanged.

Targets release/17.0 so the feature can ship with 17.0. It depends on an Aspire CLI change, tracked in microsoft/aspire#20967 (see below).

• StaticHost remains the public entry point and forwards every on-demand API request to Astro. The two API landing pages (the heaviest and most visited) stay prerendered. Astro's native memory provider is the only response-cache tier: 128 entries per renderer, 30-minute retention. It is count-bounded, not byte-bounded.
• Request handling: query strings and Authorization headers are not forwarded for pages, so they cannot multiply or bypass cached renders. Concurrent requests for one cold page share a single render. Pages are routed to a renderer by a stable hash of their path, so each worker keeps its own half of the cache warm; a failed renderer is skipped for 10 seconds but kept as a last resort. Responses carry a weak ETag derived from the build id and path with Cache-Control: no-cache, and a matching If-None-Match is answered with 304 without calling the renderer. Unknown API URLs get the site 404 page, and 502/504 get a retry page with Retry-After.
• Two private renderers provide matching-build failover. Each verifies an actual API render before becoming ready. StaticHost buffers complete responses up to 64 MiB, retries GETs with 10-second attempt limits inside a 30-second budget, and exposes /healthz/api-docs. Losing both renderers produces explicit API gateway errors while static pages remain available; App Service health checks use /healthz so a renderer outage does not make the instance unhealthy, while warmup still waits for renderer readiness. Renderers run with a 1 GiB heap limit.
• The selected Azure model uses a .NET main container and two separate Astro sidecar containers in the same App Service, with two platform instances, Always On, managed-identity image pulls, and renderer-aware health and warmup settings. Sidecars do not inherit app secrets or connection strings.
• The images have independent filesystems and no shared application volumes. .NET publishes only its application, static client and routing manifest. Dockerfile.hybrid packages the Astro server, production runtime dependencies, local image inputs and highlighting metadata. Supply a prebuilt immutable renderer image through api-docs-image; Aspire publishes the .NET image. Both images must come from the same Linux artifact. HybridApiDocsArtifactDirectory selects an artifact outside dist-hybrid.
• Compression: hybrid mode serves plain files and forwards uncompressed renderer output, so it lost the build-time Brotli of the static build. StaticHost now applies Brotli/gzip response compression (fastest level) to the hybrid pipeline. Measured locally against what production serves precompressed: the TypeScript API landing page goes from 13.9 MB to 0.98 MB (production 0.62 MB), the homepage from 302 KB to 82 KB (41 KB), and sitemap-0.xml from 14.5 MB to 752 KB (176 KB). The trade-off is per-request CPU (about 20 ms for the 13.9 MB page at Brotli quality 1) and output roughly 1.5x to 4x larger than build-time maximum quality. Precompressing the hybrid client would close that gap.
• Existing route aliases, canonical redirects, Markdown negotiation, API search and sitemap discovery are preserved. CI compares emitted static/hybrid output and exercises actual independent containers, native MISS/HIT behavior, browser assets, failover, total outage and recovery.

Aspire CLI dependency

aspire docs api list|search|get builds its catalog only from https://aspire.dev/sitemap-0.xml. In hybrid mode Astro's generated sitemap-0.xml no longer lists the API routes, because they are not prerendered (only the 2 prerendered landing pages remain, instead of about 24,000 URLs). The build therefore writes a separate sitemap-api.xml with every on-demand API HTML URL (about 24,200) and links it from sitemap-index.xml.

The CLI must ingest the catalog from the sitemap index. That change, the exact ingestion steps and an open question about older CLIs are in microsoft/aspire#20967. Until it ships, list and search against a hybrid-built site return almost nothing, while get for a known page works.

The model was deployed to Azure in a test subscription (West US 2) with a single aspire deploy after a one-time step that puts the renderer image in the registry. It runs as a .NET main container plus two Astro sidecars on a P0v3 plan, shaped like production (elastic scale, 2 minimum and 1 pre-warmed instance); two instances were ready, both renderers started on ports 4323 and 4324 on each, and the homepage, static and on-demand API pages, the search index, Pagefind, terminal recordings, the 404 page and /healthz/api-docs all returned the expected responses. The first deploy exposed a real bug: App Service treats a sidecar environment variable's value as the name of an app setting, so the literal PORT values were ignored, both renderers bound 4323 and the whole site returned 503. Each sidecar now gets its port from its own app setting, with a test. Still unverified: automatic scale-out beyond the minimum, slot swaps with sidecars, behavior under production load, and the Azure DevOps and EV2 pipelines, which need a Linux build job and a second image-push step.

Measured POC result

The native-cache acceptance run on 502c9419 passed using the process-based harness, before the independent-container packaging revision:

It avoided 48,451 static API HTML/Markdown files and preserved 8,828 non-API outputs. The private server bundle was 123,256,415 bytes. These are measurements of that revision, not the latest container images; hybrid mode still had higher peak build memory.

Third-party links and affiliations

None. This change does not add or change third-party links.

Validation

Independent containers. Run 38416394011 built the static baseline, the hybrid artifact and both images, and passed output parity, but the container check failed because the Astro image was missing runtime dependencies. Commit 55f9ba5b fixed that. Run 38426759110 rechecked the images against that run's Linux artifact and passed: separate containers with no mounts and a shared network namespace, native cache MISS then HIT, primary failover, total renderer loss, and recovery. Commits after 55f9ba5b (45d688f9) are covered by the unit and .NET checks below but not by a full CI run.

Merged result on release/17.0. The merge of this branch with release/17.0 (31 newer commits) is clean. In a throwaway worktree of that merge:

• From src/frontend: pnpm install --frozen-lockfile --prefer-offline — passed.
• From src/frontend: pnpm exec astro sync — passed.
• From src/frontend: pnpm exec vitest run --config vitest.config.ts tests/unit/hybrid-api-docs.vitest.test.ts tests/unit/hybrid-native-cache.vitest.test.ts tests/unit/api-markdown.vitest.test.ts tests/unit/ts-api-route-catalog.vitest.test.ts tests/unit/build-performance.vitest.test.ts — passed, 61 tests and one existing skip.
• From the repository root: dotnet test .\tests\Aspire.Dev.AppHost.Tests\Aspire.Dev.AppHost.Tests.csproj --nologo --verbosity minimal — passed, 11 tests, including generated Bicep, sidecar topology, managed identity, ports and readiness settings.
• From the repository root: dotnet test .\tests\StaticHost.Tests\StaticHost.Tests.csproj --nologo --verbosity minimal --filter 'FullyQualifiedName!~Redis' — passed, 315 tests.

Compression fix. Commits ecbdc0cd and 5f6063c0 add the response compression above. From the repository root: dotnet test .\tests\StaticHost.Tests\StaticHost.Tests.csproj --nologo --verbosity minimal --filter 'FullyQualifiedName!~Redis' — passed, 319 tests (including Brotli and gzip round-trips for forwarded and static responses); dotnet test .\tests\Aspire.Dev.AppHost.Tests\Aspire.Dev.AppHost.Tests.csproj --nologo --verbosity minimal — passed, 11 tests. Both ran in a throwaway worktree of this branch rather than the release/17.0 merge. Sizes above were measured on the local preview after rebuilding StaticHost.

Earlier local checks. pnpm exec eslint on the changed hybrid files passed. pnpm exec tsc --noEmit --pretty false fails on existing repository diagnostics, with none reported in the changed hybrid files. The preceding native-cache CI run passed Linux acceptance and Windows artifact/contracts checks, including pnpm exec playwright test --config playwright.hybrid.config.mjs. A local isolated Aspire preview using the Windows CI artifact served the homepage, API pages and /healthz/api-docs with both renderers healthy.

API search indexes: the landing, package and module pages embedded their search index in the HTML (13.3 MB on the TypeScript landing, in production too). They now fetch prerendered /reference/api-search/{csharp,typescript}[/slug].json files (outside reference/api/, so they stay static in hybrid mode), with a delayed "Loading search index…" state and an in-place Try again button when the download fails. The TypeScript landing HTML drops from 13.6 MB to 298 KB and all API pages from 105.6 MB to 70.6 MB; the type/item pages keep their tiny inline indexes. All 334 generated indexes (95,333 entries) are byte-identical to the previous inline ones. Validated with eslint, the full frontend unit suite (80 files, 1,025 tests) and the affected desktop Chromium e2e tests (11 passed). verify-hybrid-output.mjs now asserts the index files stay prerendered.
Hardening from a design review. Measured against a local preview of the Windows CI artifact: query strings and Authorization forced a render on every request (now cache hits of about 8 ms); 12 concurrent requests for one cold page caused 12 renders (now 1); a burst of 24 renders raised a cache hit from 8 ms to 221 ms (now a median of 9 ms); 80 distinct pages split 45/35 across the two renderers; revalidation returns 304 with no renderer request; and unknown API URLs return the 157 KB site 404 page instead of an empty body. The landing pages were the one expensive case: 40 concurrent landing renders took a 5.2 s median, grew a worker to 1.4 GB RSS and pushed a health probe to the 2 s timeout. With the landings prerendered the same flood takes a 0.5 s median, worker memory stays near 440 MB, and the health probe peaks at 140 ms. Package, module, type and member pages render in a median of 22-31 ms, so they stay on demand. Run 38503203899 passed the whole hybrid pipeline on 1ffec64c, including output parity (landings prerendered, everything else on demand), the independent-container acceptance script (stampede, ETag 304, query and credential handling, 404 page, retry page, failover by path ownership, total outage with /healthz staying 200, and recovery) and the browser tests. Locally: dotnet test for StaticHost (329, non-Redis) and AppHost (11) in a throwaway worktree, the hybrid frontend unit tests, and eslint on the changed files.
Outstanding: the Aspire CLI change in microsoft/aspire#20967; Azure deployment, cloud restart behavior and production load were not tested; the aspire docs api commands were not run against a hybrid site. The broader frontend gate on this PR had an unrelated failure in dashboard-image-references.vitest.test.ts (a volume-mount example) when it ran against main; neither that test nor dashboard documentation changed in this PR. No full-site build was run locally.